CHAPTER 14
Building an XMPP Application
In this chapter, we start with an existing application and look at different ways of XMPPenabling it with techniques you’ve learned earlier in the book.
The CheshiR Microblogging Platform
Meet “CheshiR,” our very own “microblogging” service. From the CheshiR website,
you can post short pieces of text about what you are currently doing, reading, or thinking. Other people can then subscribe to (or “follow”) your postings, which means that
they will receive a notification any time you post a new update. Figure 14-1 shows the
home page of a CheshiR user with three contacts.
The CheshiR service is implemented in Python, using a very simple design. On one
side, there is the HTTP frontend, which serves the web pages of the service. Whenever
a user posts a message, it is sent to the CheshiR backend, which stores all the messages
and subscriptions in a database. To display the homepage of a user, the frontend asks
the backend for all messages sent by that user’s contacts.
To extend the reach of our service and improve the experience of our users, we are
going to investigate different ways of integrating XMPP into our platform. Because we
like the principles of agile software development, we’ll take an iterative approach, with
focused code sprints to add a few well-defined features during each sprint.
First Sprint: The CheshiR XMPP IM Bot
Analysis
Having a web interface to post messages on CheshiR is convenient for our users, because it avoids the need to install a special application to read or post messages to their
microblog. However, the downside of a web interface is that our users constantly need
to switch to their web browser and hit the “Reload” button to find out whether new
messages have arrived. Not only is such a polling system inconvenient for the user, but
211
Figure 14-1. Posting and receiving microblogs via the CheshiR web interface
it also puts a high load on the service itself (see Chapter 8). Each time a user reloads
the web page to see if new messages are waiting, the HTTP frontend of our service
needs to query the backend database to render the “messages page” for that user. Since
this page doesn’t change that often (at best, a few new messages have arrived, which
still leaves the rest of the page unchanged), this means a lot of data is needlessly requested over and over again. Moreover, this approach won’t scale over the long term:
the more users our service attracts, the more requests for new messages our service will
need to process. We’re already starting to worry that our web server will eventually
succumb to the high load of requests, so we do some research about technologies that
might help us send a notification whenever a user posts to her microblog.
Design
Our research indicates that a technology called XMPP might help to solve the user’s
inconvenience  and  the  server’s  scalability  problem.  We  decide  that  a  simple  XMPP
“frontend” might enable our users to interact with their microblogs from within the
comfort of their IM clients (or specialized microblogging clients). What’s more, the
service will automatically push new messages to the connected XMPP clients. Whenever
212 | Chapter 14: Building an XMPP Application
a notification arrives at the IM client, the user will be notified of the new message,
removing the need for the user to regularly poll for new messages. And with users no
longer constantly polling for new messages, the heavy load of handling the page requests in the HTTP frontend will also be lifted.
Our small team of developers gathers around the whiteboard to sketch out the architecture shown in Figure 14-2. Whereas the web-based approach always sends requests
from the browser to the HTTP frontend through to the backend, the XMPP approach
also has events going in the other direction. Besides being able to interact with the web
service from a standard IM client, XMPP also provides a standard interface for communicating with dedicated, custom CheshiR applications, tailored specifically toward
microblogging.
Figure 14-2. The architecture of the CheshiR service: web browsers query the HTTP frontend, which
in turn queries the backend; IM and custom clients post messages to the backend through the XMPP
interface, while the backend notifies the XMPP clients of incoming messages when they arrive
For our first sprint we decide to design the CheshiR frontend as a simple bot (similar
to the basic “echo” XMPP service discussed in Chapter 2). Users can add the bot as a
contact to their existing rosters, and interact with it the same way they interact with
regular contacts. Whenever a user sends a message to the bot, the message is forwarded
to the microblog of the user in question. Conversely, if someone that the user is following posts a message, the bot sends this message to the user’s IM client. Besides
allowing our users to post to the microblog by sending a message to the bot, we also
want the bot to automatically post all the <status/> text that users post in their presence
updates (e.g., “in a meeting” or “stepped out for lunch”). This gives users of our XMPP
First Sprint: The CheshiR XMPP IM Bot
| 213
service the extra convenience of sharing their status on their microblog and on their IM
network with only one operation.
We quickly sketch out what the user’s interaction will look like—pretty much standard
IM conversations, as shown in Figure 14-3.
Figure 14-3. Posting and receiving microblogs via the CheshiR IM bot
Coding
Now that we have the basic design set, we do a bit of research into off-the-shelf XMPP
libraries. Because we find that the Python language is great for fast prototyping as well
as high-scale application development (as mentioned, our backend is written in Python), we check out several of the available Python libraries. After looking at APIs and
developer support, we select SleekXMPP (the same library used for the echo bot in
Chapter 2).
214 | Chapter 14: Building an XMPP Application
Once  we  settle  on  the  library  to  use,  getting  the  bot  implemented  is  rather
straightforward. The basic code for our implementation is encapsulated in a Bot class,
as you can see in Example 14-1.
Example 14-1. CheshiR IM bot implementation
class Bot :
  def __init__(self, jid, password, backend, url) :
    self.url = url
    self.xmpp = sleekxmpp.ClientXMPP(jid, password)
    self.xmpp.add_event_handler("session_start", self.handleXMPPConnected)
    for event in ["message", "got_online", "got_offline", "changed_status"] :
      self.xmpp.add_event_handler(event, self.handleIncomingXMPPEvent)
    self.backend = backend
    self.backend.addMessageHandler(self.handleMessageAddedToBackend)
  def handleXMPPConnected(self, event):
    self.xmpp.sendPresence()
  def handleIncomingXMPPEvent(self, event) :
    message = event["message"]
    user = self.backend.getUserFromJID(event["jid"])
    self.backend.addMessageFromUser(message, user)
  
  def handleMessageAddedToBackend(self, message) :
    body = message.user + ": " + message.text
    htmlBody = "<a href=\"%(uri)s\">%(user)s</a>: %(message)s" % {
      "uri": self.url + "/" + message.user,
      "user" : message.user, "message" : message.text }
    for subscriberJID in self.backend.getSubscriberJIDs(message.user) :
      self.xmpp.sendMessage(subscriberJID, body, mhtml=htmlBody)
  
  def start(self) :
    self.xmpp.connect()
    self.xmpp.process()
The functions are as follows:
• In the constructor of Bot, add_event_handler is called to register the bot to handle
several XMPP events: incoming XMPP messages, users changing their XMPP presence, and the start of the XMPP session (right after the authentication process has
finished).
• addMessageHandler registers our bot to be notified of messages added to the backend. These messages could have been added through the web interface or the XMPP
interface.
• In the handleXMPPConnected function, the bot will send initial presence whenever it
logs in (e.g., after being disconnected if the server goes down); this will make the
bot pop up in the roster of all our subscribers.
First Sprint: The CheshiR XMPP IM Bot
| 215
• In  the  handleIncomingXMPPEvent  function,  any  incoming  XMPP  message  will  be
mapped to a user of the blog platform, and the new message will be submitted to
the backend for storage.
• In the handleMessageAddedToBackend function, the bot will generate an outbound
XMPP message to all of the user’s followers. This message contains two main child
elements: the plain text of the notification is included in the XMPP <body/> child
of the message stanza, and a formatted version of the notification is included using
XHTML-IM. The formatted version includes auto-generated links to the CheshiR
homepage of the person who posted the message.
To add the bot to CheshiR, we simply instantiate it at the start of our application, as
shown in Example 14-2. The bot needs an XMPP account at a server, which it logs into
on startup. CheshiR users can then add this bot as a contact in their contact list, and
send messages to it.
Example 14-2. Instantiating the CheshiR IM bot upon application start
def main() :
  backend = SimpleBackend()
  bot = Bot("bot@cheshir.lit", "mypass", backend, "http://cheshir.lit")
  bot.start()
  httpFrontend = HTTPFrontend(8080, backend)
  httpFrontend.start()
With less than 30 lines of code, we were able to create a simple, generic interface to
our microblogging platform!
Second Sprint: Configuring the CheshiR XMPP IM Bot
Analysis
We now have a bot that monitors both incoming messages and users’ status changes,
and then posts these to the CheshiR service. After a few weeks, some users complain
that their feeds get filled with “Auto Status (idle)” messages, just because their IM client
automatically  changes  the  status  message  after  long  periods  of  inactivity.  A  bit  of
brainstorming in the developer room leads us to think that could let users disable the
automatic posting of presence changes to their microblog. Either way, account configuration methods are in order, so we head back to the whiteboard for a design session.
Design
We envision several possibilities:
• Enable users to configure their accounts via the website.
216 | Chapter 14: Building an XMPP Application
• Force  users  to  send  a  magic  message  to  the  CheshiR  bot,  such  as  disable_sta
• Use a native XMPP configuration method, such as ad-hoc commands (see Chap
tus_monitoring.
ter 11).
Because we’re really starting to get into this XMPP stuff, we decide to experiment with
the ad-hoc commands approach.
Coding
Fortunately, the library we’re using makes it easy to add support for ad-hoc commands.
With only a few more lines of code, we’re able to provide an interface to the users’
settings. There are five notable changes:
• In the constructor, registerPlugin is called to load support for the three XEPs we
need: Data Forms [XEP-0004], Service Discovery [XEP-0030], and Ad-Hoc Commands [XEP-0050].
• Also in the constructor, there are now separate handlers for incoming messages
incoming  presence  changes,  namely  handleIncomingXMPPEvent  and
and 
handleIncomingXMPPPresence, respectively.
• The last change to the constructor is that a form is created with the Data Forms
plug-in and then registered with the Ad-Hoc Commands plug-in, to allow the users
to query the command and run it.
• In the new presence change handler handleIncomingXMPPPresence, the bot now calls
the backend implementation to check whether a user’s presence should be monitored for changes before it creates a new post from it.
• The 
last  change 
is  the  new  handler  for  the  configuration  command,
handleConfigurationCommand, which reads the new value out of the submitted form
and updates the configuration in the backend.
These changes are highlighted in Example 14-3.
Example 14-3. Configurable CheshiR IM bot implementation, with the differences from the basic IM
bot shown in Example 14-1
class ConfigurableBot :
  def __init__(self, jid, password, backend, url) :
    self.url = url
    self.xmpp = sleekxmpp.ClientXMPP(jid, password)
    for plugin in ["xep_0004", "xep_0030", "xep_0050"] :
      self.xmpp.registerPlugin(plugin) 
    self.xmpp.add_event_handler("session_start", self.handleXMPPConnected)
    self.xmpp.add_event_handler("message", self.handleIncomingXMPPEvent)
    for event in ["got_online", "got_offline", "changed_status"] :
      self.xmpp.add_event_handler(event, self.handleIncomingXMPPPresence)
    self.backend = backend
    self.backend.addMessageHandler(self.handleMessageAddedToBackend)
Second Sprint: Configuring the CheshiR XMPP IM Bot
| 217
    configurationForm = self.xmpp.plugin["xep_0004"].makeForm("form", "Configure")
    configurationForm.addField(
        var="monitorPresence", label="Use my status messages",
        ftype="boolean", required=True, value=True)
    self.xmpp.plugin["xep_0050"].addCommand("configure", "Configure",
        configurationForm, self.handleConfigurationCommand)
  def handleConfigurationCommand(self, form, sessionId):
    values = form.getValues()
    monitorPresence = True if values["monitorPresence"] == "1" else False
    jid = self.xmpp.plugin["xep_0050"].sessions[sessionId]["jid"]
    user = self.backend.getUserFromJID(jid)
    self.backend.setShouldMonitorPresenceFromUser(user, monitorPresence)
  def handleIncomingXMPPPresence(self, event):
    user = self.backend.getUserFromJID(event["jid"])
    if self.backend.getShouldMonitorPresenceFromUser(user):
      self.handleIncomingXMPPEvent(event)
  def handleXMPPConnected(self, event):
    self.xmpp.sendPresence()
  def handleIncomingXMPPEvent(self, event):
    message = event["message"]
    user = self.backend.getUserFromJID(event["jid"])
    self.backend.addMessageFromUser(message, user)
  
  def handleMessageAddedToBackend(self, message) :
    body = message.user + ": " + message.text
    htmlBody = "<a href=\"%(uri)s\">%(user)s</a>: %(message)s" % {
      "uri": self.url + "/" + message.user,
      "user" : message.user, "message" : message.text }
    for subscriberJID in self.backend.getSubscriberJIDs(message.user) :
      self.xmpp.sendMessage(subscriberJID, body, mhtml=htmlBody)
  
  def start(self) :
    self.xmpp.connect()
    self.xmpp.process()
Third Sprint: Scaling the CheshiR XMPP Service Using a Server
Component
Analysis
A few more weeks pass by, and we notice that our XMPP service is becoming rather
slow. Sometimes it even stops working due to a lack of resources. After some investigation, we discover that it is actually our bot that is causing some havoc. It seems that
our XMPP interface has become very popular—in fact, the bot’s roster is so large that
the server cannot handle it any longer.
218 | Chapter 14: Building an XMPP Application
The reason for this scenario is that many XMPP server implementations are not optimized to deal with huge rosters like the one from our bot. (All XMPP servers reach such
a  limit  eventually,  because  they  simply  are  not  designed  for  rosters  with  10,000  or
100,000 items!) Because we’ve run into a limit on the number of CheshiR users who
can interact with the bot, we need to find a more scalable way of providing the hot new
XMPP interface to our users.
So we ask ourselves: why does our XMPP server need to keep track of the bot’s roster
in the first place? Our CheshiR backend knows who signed up for the XMPP service,
so why does this information need to be duplicated in the XMPP server’s database? The
root cause is presence routing, which is described in detail in Chapter 3. Whenever our
bot generates a presence notification, our server needs to distribute that information
to every user of the XMPP service. Additionally, whenever one of our users connects
to the XMPP network, it will query the presence of our bot (using a presence probe),
and our bot’s XMPP server will automatically answer to this with the current presence
of the bot.
However, if we could offload control of presence-related functionality from the server,
maybe we would no longer need the server to keep track of the bot’s roster. It’s time
to head back to the whiteboard and see what we can come up with....
Presence Scaling in Components
In fact, it’s not the presence that’s the problem, per se, but the roster.
When a bot logs into the server and requests the roster, the server must
fetch the entire roster and send it as one giant stanza to the bot. Although
it’s probably vain to believe that we can handle this better than the server
can, we can improve matters by not handling it at all. Because, unlike
our bot, a component isn’t tied to this atomic roster get, the roster can
be loaded from disk lazily, or in stages, and as such avoid the load of a
(potentially)  multi-hundred-megabyte  roster  stanza.  The  load  saved
from not probing the presence of our contacts is also significant, as our
component  need  not  know  the  contacts’  status  on  login  and  will  be
happy to see presence stanzas trickle in over time.
Design
To solve our scalability problems, we do a bit more research into XMPP and discover
that most XMPP servers are architected in a modular way that enables the server to
offload stanza processing onto an external component. This approach seems like it
might solve our scalability issues, so we decide to transform our CheshiR bot into an
XMPP server component. This will require some code changes as well as some configuration changes to our XMPP server so that our server will allow our new component
to connect as a trusted part of the system. As described in Chapter 13, this means that
our component will be assigned a predefined subdomain of the server, and will directly
receive all stanzas addressed to this domain without the server taking any other actions.
Third Sprint: Scaling the CheshiR XMPP Service Using a Server Component
| 219
This, of course, means that our component will need to handle presence broadcasts
and probe responses on its own, which might be just what we need to make the system
more scalable.
Coding
Our design decisions lead us to refactor the bot code so that it will run as a component.
However,  it  turns  out  that  we  don’t  need  to  make  too  many  changes.  Handling  of
incoming  presence,  message  events,  and  messages  added  from  the  web  interface
behaves exactly the same as before. The main difference is that now everything related
to presence broadcasting and presence subscriptions needs to be handled by the component itself. Here’s an overview of the modifications:
• Because initial presence broadcasting is no longer handled by the server, the component needs to send out its presence to all its users in handleXMPPConnected.
• Whenever  an  XMPP  user  probes  for  the  presence  of  our  XMPP  service,  handle
XMPPPresenceProbe sends back the presence of the component (which in this case,
is always “available”).
• When 
subscribes 
an  XMPP  user 
component’s  presence,
handleXMPPPresenceSubscription  automatically  authorizes  the  subscription  request, and sends the current presence of the component to the user. Additionally,
because the component wants to listen to presence changes from CheshiR user, it
also sends a subscription request to the originating user.
the 
to 
These modifications are shown in Example 14-4, which you can compare to the simple
bot code in Example 14-1.
To deploy these changes, we just need to replace our bot with a server component in
our application, as shown in Example 14-5.
Example 14-4. CheshiR IM server component implementation (the most important differences with
the basic bot implementation from Example 14-1 are highlighted)
class SimpleComponent :
  def __init__(self, jid, password, server, port, backend) :
    self.xmpp = sleekxmpp.componentxmpp.ComponentXMPP(jid, password, server, port)
    self.xmpp.add_event_handler("session_start", self.handleXMPPConnected)
    self.xmpp.add_event_handler("changed_subscription", 
        self.handleXMPPPresenceSubscription)
    self.xmpp.add_event_handler("got_presence_probe", 
        self.handleXMPPPresenceProbe)
    for event in ["message", "got_online", "got_offline", "changed_status"] :
      self.xmpp.add_event_handler(event, self.handleIncomingXMPPEvent)
    self.backend = backend
    self.backend.addMessageHandler(self.handleMessageAddedToBackend)
  def handleXMPPConnected(self, event) :
    for user in self.backend.getAllUsers() :
      self.xmpp.sendPresence(pto = self.backend.getJIDForUser(user))
220 | Chapter 14: Building an XMPP Application
  def handleIncomingXMPPEvent(self, event) :
    message = event["message"]
    user = self.backend.getUserFromJID(event["jid"])
    self.backend.addMessageFromUser(message, user)
  def handleXMPPPresenceProbe(self, event) :
    self.xmpp.sendPresence(pto = self.backend.getJIDForUser(user))
  def handleXMPPPresenceSubscription(self, subscription) :
    if subscription["type"] == "subscribe" :
      userJID = subscription["from"]
      self.xmpp.sendPresenceSubscription(pto=userJID, ptype="subscribed")
      self.xmpp.sendPresence(pto = userJID)
      self.xmpp.sendPresenceSubscription(pto=userJID, ptype="subscribe")
  def handleMessageAddedToBackend(self, message) :
    body = message.user + ": " + message.text
    for subscriberJID in self.backend.getSubscriberJIDs(message.user) :
      self.xmpp.sendMessage(subscriberJID, body)
  def start(self) :
    self.xmpp.connect()
    self.xmpp.process()
Example 14-5. Instantiating the CheshiR IM server component upon application start
def main() :
  backend = SimpleBackend()
  component = SimpleComponent(
    jid = "component.cheshir.lit", password = "mypass",
    server = "cheshir.lit", port = 5060, backend = backend)
  component.start()
  httpFrontend = HTTPFrontend(8080, backend)
  httpFrontend.start()
We now have an XMPP service that no longer puts the burden of roster and presence
management on the XMPP server. As a result, our service is more scalable. From the
perspective of our users, the only thing that has changed is that they now need to add
posts.cheshir.lit to their roster instead of bot@cheshir.lit. (Some XMPP servers can
redirect traffic for a particular user@domain.tld address to a component, so even this
might not be necessary.)
Fourth Sprint: Registering with the CheshiR Server Component
Analysis
So far, we’ve assumed that our users would sign up for the XMPP service from the
CheshiR web page, where they would provide the JID they would use for posting and
receiving messages. But we know that some XMPP clients enable you to register with
Fourth Sprint: Registering with the CheshiR Server Component
| 221
services from within the client itself, using in-band registration as described in “What’s
in a Nick?” on page 85. Adding this feature sounds like it might make our service even
easier to use and further differentiate it from the crowded microblogging field, so we
decide to investigate the possibility.
Design
To  provide  in-band  registration,  we  look  for  a  plug-in  in  SleekXMPP  that  provides
jabber:iq:register 
the
documentation and source code, we conclude that SleekXMPP does not yet have support for in-band registration (although that might have changed by the time you read
this book!). So we will have to write our own custom handling for IQ requests for the
jabber:iq:register namespace. Some hacking is in order....
functionality.  Unfortunately,  after 
through 
looking 
We envision a registration screen of the kind in Figure 14-4.
Figure 14-4. The CheshiR registration form, as presented in an IM client
Coding
To make this happen, it seems that we need to add a few new pieces of code to our
component:
1. First, we need the xep_0030 plug-in, which allows us to announce support for the
jabber:iq:register  extension  in  the  service  discovery  information  of  our
component.
2. Next,  we  register  handlers  for  both  get  and  set  IQ  requests  for  the
jabber:iq:register namespace. This way, whenever a user’s client wants to register with the service, it will first send an IQ-get request to find out what information
it  needs  to  provide  to  the  service.  Upon  receiving  such  a  request,
222 | Chapter 14: Building an XMPP Application
handleRegistrationFormRequest  sends  back  a  response,  adding  a  username  and
password  element,  indicating  that  it  expects  this  information  when  the  user
registers.
3. After the user has entered the requested information, the user’s client submits the
registration form by embedding the filled form in an IQ-set request to the component, which is processed by handleRegistrationRequest.
4. After having extracted the username and password from the received form, the
component checks whether registration with these credential succeeds. If it does,
a successful (empty) result is sent back, acknowledging successful registration. If
registration fails (for example, because of an incorrect password), the component
informs the user of the problem by sending back an IQ reply with an error payload.
These changes are shown in Example 14-6.
Example 14-6. CheshiR IM server component with in-band registration support (only the changed
and new methods since Example 14-5 are shown)
class RegistrableComponent :
  def __init__(self, jid, password, server, port, backend) :
    self.xmpp = sleekxmpp.componentxmpp.ComponentXMPP(jid, password, server, port)
    self.xmpp.add_event_handler("session_start", self.handleXMPPConnected)
    self.xmpp.add_event_handler("changed_subscription", 
        self.handleXMPPPresenceSubscription)
    self.xmpp.add_event_handler("got_presence_probe", 
        self.handleXMPPPresenceProbe)
    for event in ["message", "got_online", "got_offline", "changed_status"] :
      self.xmpp.add_event_handler(event, self.handleIncomingXMPPEvent)
    self.backend = backend
    self.backend.addMessageHandler(self.handleMessageAddedToBackend)
    self.xmpp.registerPlugin("xep_0030")
    self.xmpp.plugin["xep_0030"].add_feature("jabber:iq:register")
    self.xmpp.add_handler("<iq type='get' xmlns='jabber:client'>" + 
      "<query xmlns='jabber:iq:register'/></iq>", self.handleRegistrationFormRequest)
    self.xmpp.add_handler("<iq type='set' xmlns='jabber:client'>" +
      "<query xmlns='jabber:iq:register'/></iq>", self.handleRegistrationRequest)
  def handleRegistrationFormRequest(self, request) :
    payload = ET.Element("{jabber:iq:register}query")
    payload.append(ET.Element("username"))
    payload.append(ET.Element("password"))
    self.sendRegistrationResponse(request, "result", payload)
  def handleRegistrationRequest(self, request) :
    jid = request.attrib["from"]
    user = request.find("{jabber:iq:register}query/{jabber:iq:register}username")
    password = request.find("{jabber:iq:register}query/{jabber:iq:register}password")
    if self.backend.registerXMPPUser(user, password, jid) :
      self.sendRegistrationResponse(request, "result")
    else :
      error = self.xmpp.makeStanzaError("forbidden", "auth")
      self.sendRegistrationResponse(request, "error", error)
Fourth Sprint: Registering with the CheshiR Server Component
| 223
  def sendRegistrationResponse(self, request, type, payload = None) :
    iq = self.xmpp.makeIq(request.get("id"))
    iq.attrib["type"] = type
    iq.attrib["from"] = self.xmpp.fulljid
    iq.attrib["to"] = request.get("from")
    if payload :
      iq.append(payload)
    self.xmpp.send(iq)
Fifth Sprint: Extending the Server Component with Rosters
Analysis
Will, the marketing guy, says we need to boost communication among our users. Right
now, all messages from a user’s contact are delivered to his client in one continuous
stream. Whenever a user wants to respond to one of his contact’s posts, he starts his
message with “@contact”, a popular convention on microblogging services. Although
this way of communicating works, marketing thinks we should be able to improve the
user experience for this, which would increase the “stickiness” of our service (whatever
that means). So once again we sketch out some ideas at the whiteboard.
Design
We  already  mentioned  that  our  server  component  gets  assigned  the  complete
posts.cheshir.lit subdomain. So far, we’ve only used the domain JID to send and
receive messages. However, because we have a whole domain at our disposal, we can
create as many arbitrary JIDs as we want. This means that people’s posts can come
from, 
just
say,  alice@posts.cheshir.lit  or  rabbit@posts.cheshir.lit,  not 
posts.cheshir.lit. Conversely, sending a message to rabbit@posts.cheshir.lit could
prefix that message with “@rabbit” before posting it to your microblog, thus simulating
a public one-to-one conversation in the microblogging style.
We can extend this feature a bit by associating the user’s presence on the XMPP network
with his alias on the CheshiR service. In particular, when we receive a presence change
from a user’s real JID, we will republish it to his CheshiR account as his latest post and
as his presence status on CheshiR. The basic idea is shown in Figure 14-5.
Coding
To implement these enhanced features, we need to change the behavior of a few of the
methods from our previous version of the component:
• Instead  of  sending  one  presence  stanza  from  the  XMPP  service  to  every  user,
handleXMPPConnected needs to send a presence stanza for each follower of every
user, containing the last message of the contact as a presence status.
224 | Chapter 14: Building an XMPP Application
Figure 14-5. Using the CheshiR server component from an XMPP client: CheshiR contacts are mixed
with regular XMPP contacts, messages from different contacts get delivered in their own chat dialog,
and the message sent to Kevin will result in the message “@kevin I know the feeling!” being posted
on Remko’s microblog
• Incoming XMPP messages that are directed to a user need to have “@user” pre
pended to them in handleIncomingXMPPMessage.
• Since the user’s server will send presence to every one of the user’s contacts, there
will be multiple incoming presence stanzas. Because we don’t want to post a presence event more than once, we’ll only listen to the presence events directed to the
component itself in handleIncomingXMPPPresence.
• handleXMPPPresenceProbe returns the last message for presence probes directed to
contacts.
• Whenever a user subscribes to our service, the component sends a presence subscription request for every contact in the user’s contact list, which results in the
contacts getting added to the user’s XMPP roster.
• Whenever a message is posted using the web interface, we send a message to all
subscribed users, and send it from the corresponding JID of the author. This delivers the message to the XMPP clients as a message from the contact.
Example 14-7 shows the updated version of the bot, which uses the helper methods
shown in Example 14-8.
Example 14-7. Extended CheshiR IM server component implementation
class Component :
  def __init__(self, jid, password, server, port, backend) :
    self.componentDomain = jid
Fifth Sprint: Extending the Server Component with Rosters
| 225
    self.xmpp = sleekxmpp.componentxmpp.ComponentXMPP(jid, password, server, port)
    self.xmpp.add_event_handler("session_start", self.handleXMPPConnected)
    self.xmpp.add_event_handler("changed_subscription",
        self.handleXMPPPresenceSubscription)
    self.xmpp.add_event_handler("got_presence_probe",
        self.handleXMPPPresenceProbe)
    self.xmpp.add_event_handler("message", self.handleIncomingXMPPMessage)
    for event in ["got_online", "got_offline", "changed_status"] :
      self.xmpp.add_event_handler(event, self.handleIncomingXMPPPresence)
    self.backend = backend
    self.backend.addMessageHandler(self.handleMessageAddedToBackend)
  def handleXMPPConnected(self, event) :
    for user in self.backend.getAllUsers() :
      self.sendPresenceOfAllContactsForUser(user)
  def handleIncomingXMPPMessage(self, event) :
    message = self.addRecipientToMessage(event["message"], event["to"])
    user = self.backend.getUserFromJID(event["jid"])
    self.backend.addMessageFromUser(message, user)
  def handleIncomingXMPPPresence(self, event) :
    if event["to"] == self.componentDomain :
      user = self.backend.getUserFromJID(event["jid"])
      self.backend.addMessageFromUser(event["message"], user)
  def handleXMPPPresenceProbe(self, event) :
    self.sendPresenceOfContactToUser(contactJID=event["to"], userJID=event["from"])
  def handleXMPPPresenceSubscription(self, subscription) :
    if subscription["type"] == "subscribe" :
      userJID = subscription["from"]
      user = self.backend.getUserFromJID(userJID)
      contactJID = subscription["to"]
      self.xmpp.sendPresenceSubscription(
          pfrom=contactJID, pto=userJID, ptype="subscribed", pnick=user)
      self.sendPresenceOfContactToUser(contactJID=contactJID, userJID=userJID)
      if contactJID == self.componentDomain :
        self.sendAllContactSubscriptionRequestsToUser(userJID)
  def handleMessageAddedToBackend(self, message) :
    userJID = self.getComponentJIDFromUser(message.user)
    for subscriberJID in self.backend.getSubscriberJIDs(message.user) :
      self.xmpp.sendMessage(mfrom=userJID, mto=subscriberJID, mbody=message.text)
      self.xmpp.sendPresence(pfrom=userJID, pto=subscriberJID, pstatus=message.text)
Example 14-8. CheshiR IM server component helper methods
  def sendPresenceOfAllContactsForUser(self, user) :
    userJID = self.backend.getJIDForUser(user)
    for contact in self.backend.getContacts(user) :
      contactJID = self.getComponentJIDFromUser(contact)
      self.sendPresenceOfContactToUser(contactJID = contactJID, userJID = userJID)
  def sendPresenceOfContactToUser(self, contactJID, userJID) :
    message = self.backend.getLastMessage(contactJID).text
226 | Chapter 14: Building an XMPP Application
    self.xmpp.sendPresence(pto = userJID, pfrom = contactJID, pshow = message)
  def sendAllContactSubscriptionRequestsToUser(self, userJID) :
    user = self.backend.getUserFromJID(userJID)
    for contact in self.backend.getContacts(user) :
      contactJID = self.getComponentJIDFromUser(contact)
      self.xmpp.sendPresenceSubscription(
          pfrom=contactJID, pto=userJID, ptype="subscribe", pnick=contact)
  
  def addRecipientToMessage(self, message, recipientJID) :
    contact = self.getUserFromComponentJID(recipientJID)
    return ("@" + contact if contact else "") + " " + message
  def getUserFromComponentJID(self, jid) :
    return jid.split("@",1)[0] if "@" in jid else None
  def getComponentJIDFromUser(self, user) :
    return user + "@" + self.componentDomain
  def start(self) :
    self.xmpp.connect()
    self.xmpp.process()
Future Sprints
A living service like CheshiR doesn’t stand still. We’re already thinking about additional
enhancements. Here are two intriguing possibilities:
• We’ve heard a lot about the XMPP PubSub technology and how it can overcome
scalability issues. Perhaps it would make sense to re-architect the XMPP frontend
as a PubSub service, providing a PubSub interface for custom clients out there. In
fact, we could probably even replace the CheshiR backend with a PubSub service,
storing  all  posts  and  subscription  information  in  PubSub  nodes  and  their
subscriptions.
• The micromessaging model behind CheshiR and other microblogging services has
some intriguing similarities to groupchat. What if we created a personalized MultiUser Chat room for each user, where they could interact with their subscribers in
a familiar groupchat interface? The room occupants could even be aliases because
we don’t want to force everyone to park in multiple groupchat rooms.
Unfortunately we don’t have space to explore these potential sprints in detail, so we
leave them as an exercise for the reader.
A CheshiR Server Module or Dedicated Server?
So far, we have seen how to integrate XMPP into an application by building either a
bot or a server component. Both approaches yield a solution that is independent from
the server you are using to deploy your application. This is mostly an advantage, but
A CheshiR Server Module or Dedicated Server?
| 227
the downside of having an external XMPP integration is that you don’t have access to
the server internals, and this can limit both the functionality and the performance of
your application. In fact, our CheshiR server component has started to take on the same
kind of presence features that are usually the responsibility of an XMPP server, and it’s
likely that the server developers will do a better job of this than component developers
will!
In the case of CheshiR, we could make use of a tight integration with an XMPP server
by directly manipulating the rosters of our contacts. As a result, we wouldn’t have to
send subscription requests back and forth whenever a user registers with the service,
or whenever a user subscribes to one of his contact’s notifications.
However, building a CheshiR server module would be very specific to a given server
codebase. In particular, this level of integration would require us to code directly to the
server’s internal API, rather than using a wire protocol to communicate, so exploring
this  solution  will  take  us  outside  the  realm  of  XMPP  itself  and  into  the  world  of  a
particular codebase. Similarly, we could go even further and build our own dedicated
XMPP server implementation.
Before deciding to build a server module or a dedicated server, we would need to weigh
the costs and benefits. Is the XMPP interface really core to the CheshiR service? Can
we justify the vendor lock-in associated with writing a module for a specific server, or
the investment in building and maintaining an entirely new server codebase? Sometimes
the benefits of these paths do outweigh the costs, but only if messaging and presence
are core to your business.
Summary
In this chapter, we explored how to build an XMPP application through the example
of a microblogging system. Clearly you could make many different kinds of applications
using XMPP, and we cannot describe them all here. The thought processes and design
decisions involved might differ somewhat if you want to create, say, a voice and video
chat service, a whiteboarding client, a location tracker, a network monitoring system,
or an online gaming application. For any given type of application, you will need to
focus  more  carefully  on  certain  parts  of  the  system  (e.g.,  the  server  rather  than  the
client), write more code instead of reusing existing code, integrate more completely
with non-XMPP systems, design more XMPP extensions, or work more closely with
XMPP community. However, our intent in this book has been to give you the tools you
need to build any kind of XMPP application. If we succeeded, you should be able to
take the tools we have described and apply them in a wide range of projects, products,
and services. We wish you success in using XMPP technologies and invite you to join
the ongoing conversation in the XMPP community as more and more developers collaborate to build out the real-time Internet.
228 | Chapter 14: Building an XMPP Application
